2.2. Models
Why are the provider and model explicit?
Agent behavior, latency, context limits, tool use, and cost can change between model versions and provider adapters. The repository declares both boundaries instead of relying on SDK defaults:
model_provider: ModelProvider = ModelProvider.OPENAI_COMPATIBLE
model: str = Field(default="qwen3:4b", min_length=1)
# ``openai-compatible`` describes the ADK client contract, not the
# deployment topology. Point this URL directly at Ollama for the account-free
# first run, or at agentgateway when the governed data plane is introduced.
openai_base_url: str | None = Field(
default="http://127.0.0.1:11434/v1",
validation_alias=AliasChoices("OPENAI_BASE_URL"),
)
openai_api_key: SecretStr | None = Field(
default=SecretStr("local-ollama"),
validation_alias=AliasChoices("OPENAI_API_KEY"),
)
# Optional Gemini paths: an AI Studio API key, or Enterprise/Vertex via ADC
# with an explicit project and location.
google_api_key: SecretStr | None = Field(
default=None,
validation_alias=AliasChoices("GOOGLE_API_KEY"),
)
google_genai_use_enterprise: bool = Field(
default=False,
validation_alias=AliasChoices("GOOGLE_GENAI_USE_ENTERPRISE"),
)
google_cloud_project: str | None = Field(
default=None,
validation_alias=AliasChoices("GOOGLE_CLOUD_PROJECT"),
)
google_cloud_location: str | None = Field(
default=None,
validation_alias=AliasChoices("GOOGLE_CLOUD_LOCATION"),
)
The lockfile pins the client libraries, not a mutable Ollama tag or hosted provider weights. Record the provider, model, base URL, installed Ollama model ID where applicable, prompt version, and evaluation result together.
How does the default local model work?
The required path uses ADK's OpenAI-compatible adapter directly against Ollama:
AGENT_MODEL_PROVIDER=openai-compatible
AGENT_MODEL=qwen3:4b
OPENAI_BASE_URL=http://127.0.0.1:11434/v1
OPENAI_API_KEY=local-ollama
qwen3:4b is an Apache-2.0 open-weight model. local-ollama is a non-secret marker required by the client, not an Ollama credential. This path has no account, mandatory SaaS, or usage fee.
How does Chapter 5 add the gateway?
Keep the provider/model contract and change the base URL:
AGENT_MODEL_PROVIDER=openai-compatible
AGENT_MODEL=qwen3:4b
OPENAI_BASE_URL=http://127.0.0.1:4000/v1
OPENAI_API_KEY=local-ollama
agentgateway then owns the upstream Ollama route, traffic policy, and telemetry without forcing an application provider switch. The optional GKE profile can route to Vertex/Gemini without adding a provider-specific SDK to the OpenAI-compatible application branch.
Why require an API key for a local model?
The OpenAI client requires a non-empty key even when direct Ollama or the default local gateway route has no authentication. local-ollama is a marker, not a secret. In a real shared environment, replace that marker with actual client authentication and keep upstream cloud identity at the gateway.
How does optional native Gemini work?
Set an explicit provider and model in the gitignored root .env:
For Application Default Credentials, use GOOGLE_GENAI_USE_ENTERPRISE=true with GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_LOCATION, then run mise run doctor:gcp. Typed configuration rejects a missing or ambiguous auth path; model construction passes the chosen values explicitly to the native client and applies AGENT_MODEL_TIMEOUT_S as an HTTP request deadline. The application does not inspect key prefixes or silently switch products.
The model construction stays in one build-checked excerpt:
def build_model() -> str | BaseLlm:
"""Return the configured ADK model implementation.
Gemini mode uses ADK's native integration. The default account-free mode
uses ADK's OSS OpenAI-compatible client; ``OPENAI_BASE_URL`` chooses direct
Ollama or an agentgateway route without changing application code.
"""
if settings.model_provider is ModelProvider.GEMINI:
retry_options = _retry_options()
return Gemini(
model=settings.model,
retry_options=retry_options,
client_kwargs=_gemini_client_kwargs(retry_options),
)
if not settings.openai_base_url or not settings.openai_api_key:
raise ValueError(
"AGENT_MODEL_PROVIDER=openai-compatible requires OPENAI_BASE_URL and OPENAI_API_KEY; "
"run `mise run config:check` for the resolved configuration."
)
return ResilientOpenAILlm(
model=settings.model,
openai_base_url=settings.openai_base_url,
openai_api_key=settings.openai_api_key,
timeout_s=settings.model_timeout_s,
retries=settings.max_retries,
)
Does the course use LiteLLM?
No. Direct Ollama and agentgateway use ADK's OpenAI-compatible adapter; optional Gemini uses ADK's native adapter. The application, evaluation, and deployment paths have no LiteLLM dependency.
How should you compare models?
Hold the prompt, seed data, tool schemas, and evaluation cases constant. Compare at least:
- Exact tool trajectory success.
- Grounding and hallucination failures.
- End-to-end latency and model-call count.
- Input/output tokens and provider price where applicable.
- Policy refusal and adversarial regression behavior.
A model that writes nicer prose but chooses the wrong tool is not an upgrade for this agent.
What is the model checkpoint?
Run the staged model doctor and configuration tests, then use one read-only prompt:
Record the selected provider, model, and endpoint path. Do not compare two models from different runtime state; reset with mise run data:reset first.